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ABSTRACT 

The first 3 years of a 4-year project to plan and 
implement schoo.* change using a school improvement strategy in five 
low income, rural, resource-bound school sites are described in this 
report. One school from each of the following states — Texas, New 
Mexico, Oklahoma, Arkansas, and Louisiana — participated in the 
project. Extreme diversity characterized the schools in terms of 
population, density, minority populations, climate and topography, 
natural resources, and economies. Quantitative data analysis examined 
reading, mathematics, and language arts test scores. Qualitative 
analysis developed contextual indicators, or criteria for 
improvement, unique to each school. Each school developed its own set 
of contextual indicators and demonstrated significant changes in 
school climate. Three schools reported significant improvements in 
test scores. Findings indicate that two sets of school improvement 
standards exist: universal ones for student achievement outcomes and 
those unique to a school's goals and objectives. Recommendations are 
made for the consideration of the whole school context in assessing 
school effectiveness and for measuring school improvement in a 
variety of ways. Four tables are included. (43 references) (LMI) 
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Introduction 

Improving the effectiveness of schools in order to increase student performance 
has been a major goal of school change and improvement efforts. These efforts have 
escalated as national concern intensifies on the less-than-desirable performance of 
students currently in the nation's schools. A Nation at Risk (1983) provided an initial 
impetus for school reform. This mandate for change was followed rapidly by other 
documents and by increasing legislative action designed to address the problems of 
unsatisfactorily low student achievement. The Southwest Educational Development 
Laboratory (SEDL), one of nine federally-funded educational organizations whose 
missions are school improvement, sought to address these problems with school 
improvement projects specifically focused on rural populations. In each state of its 
five-state region (Arkansas, Louisiana, New Mexico, Oklahoma, and Texas), one 
school was identified as a demonstration school. Each demonstration school was 
small, rural, resource-bound, economically disadvantaged and interested in improving 
achievement scores. These schools varied in size of student body, size and isolation 
of district, and ethnic/minority populations. One site has primarily black students and 
one primarily Hispanic, while a third one has 50% Native American population. A 
fourth site is 60% Navajo and 20% Hispanic, while a fifth is predominately low income 
white. Two sites volunteered for the project while the other three were drafted, thus 
creating an added variable of "volunteer status" and "draftee." 

This paper summarizes and describes the first three years of a four-year project 
that focuses on planning and implementing school change utilizing an effective 
schools/school improvement strategy in the five demonstration schools. The paper 
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will: 

1 ) Review briefly the research on effective schools and school change that 
contributed to the formulation of a strategy for school improvement. 

2) Describe the data source and the situational factors in each 
demonstration school and its community: culture/customs, demographics. 

3) Review briefly information on educational indicators. 

4) Identify indicators of educational success for each school, and 
correlate each site's identified success factors to student achievement at 
that site. 

5) Finally, the authors discuss the dilemma of judging success in light of 
schools' needs to be assessed simultaneously against the universal 
standard of standardized student achievement tests, and their own 
unique goals and objectives. 

Literature Review 
Effective Schools and School Change Research 

One of the avenues by which to gain improved achievement has been the 
engagement of the effective schools research (Edmonds, 1979), one of the most 
debated and discussed phenomena in education for many years (Edmonds, 1979; 
Purkey & Smith, 1982). Edmonds and others identified factors that have been found in 
schools deemed to be effective. These factors and also concepts derived from change 
research have been used as the basis for planning and implementing school 
improvement at the five sites. A brief sampling of these two literatures follows. 

Effective Schools Factors 

The Edmonds formula (1979) for an effective school consists of five "correlates," 
characteristics or factors that appear to correlate with school effectiveness. The 
correlates are: 
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Strong leadership by the Principal in instructional matters Weber (1971) 
pointed to the instructional leadership of the building principal. Principals in effective 
schools were seen as having a clear understanding of the school's goals and mission, 
and knowing what good instruction looks like in order to develop or enhance effective 
teaching practices in their teachers. Other researchers (Armour, et al., 1976; Berman & 
;dcLaughlin, 1978; Brookover & Lezotte, 1979; Venezky & Winfield, 1979) provided 
descriptions of the behaviors of principals who are instructional leaders, such as: 
developing and implementing plans for curriculum and instruction, and expressing 
high expectaions for teachers, students and themselves. 

Systematic evaluation and monitoring of student progress. Effective schools 
monitor and measure student performance with a variety of assessment procedures. 
Principals and teachers look not only at aggregated test data, but also at a 
disaggregated analysis to make sure that all groups of students are doing well. Tests 
are analyzed to determine individual as well as group progress or problems, and this 
analysis feeds directly into instructional planning. Monitoring academic progress in 
terms of student achievement, student conduct, and student activities, for instance, are 
important to school effectiveness as revealed by Rutter, et al. (1979), Levine and Stark 
(1981), and Weber (1971). 

An orderly learning environment. A safe orderly environment means more than 
a school that is clean and free from safety hazards, although these two important 
characterisitics are part of an effective school. Teachers and students feel that, school 
is a good place to come every day and the school runs smoothly. Discipline is 
handled consistently and equitably, and the staff develops warm and positive 
relationships with students and each other. The overall climate reflects that this school 
is a good place to be, a place that is organized without being stifling, a place that is 
friendly but firm, a place where teachers and students can achieve. Such factors were 
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identified in studies by the New Yorfc State Department of Education (1974) and 
Brookover, Beady, et a). (1979). 

A clear and pervasive focus on instruction An effective school has a clear 
instructional focus. The mission of the school is instruction, and the daily schedule 
reflects this; resources and activities are directed toward instruction. There is a clear, 
comprehensive written curriculum which is part of every teacher's planning for 
instruction. Parents and community members are aware that the purpose of the school 
is instruction when hallway display cases contain samples of academic work as well 
as sports trophies. Studies by Brookover and Lezotte (1979), Trismen, Waller, and 
Wilder (1976), and others make these points. 

Hioh expectations for student achievement Another correlate is evidence Of 
high expectations for student achievement. The word "expectations" is highly related 
to observable teacher oehavior toward students: providing all students with 
appropriate questions and prompts, allowing adequate time for responses, showing 
respect to all students. Teachers and administrators establish clear rules of conduct 
and apply them consistently with aJl students. High expectations are consistently 
identified as a factor in effective schools (Rutter, et al.. 1979; Brookover & Lezotte, 
1979; Levine & Stark, 1981). 

These five correlates comprise the factors identified by Edmonds as contributing 
to effectiveness. Another factor that has received much attention in schools that are 
realizing their goals is that of parental involvement, and the SEDL strategy included 
this item for consideration by the demonstration schools. 

Support of the school by parents and community Parent involvement 
undergirds the school. An effective school mobilizes parents to become involved (this 
may mean holding parenting classes or it may mean improving the frequency and the 
quality of communications which are sent home). Research has shown a wide array of 



ways in which parents may be involved in schools (Chavkin & Williams, 1985). 
Sometimes this involvement may mean a parent tutoring program for students; it may 
mean parent volunteers working in the library. It may mean that parents come to speak 
to classes about career needs, or how they use geometry. 

These six characteristics of effective schools were the focus of staff 
development that enabled the school staffs to assess their presence or absence in 
their schools, and use such data for developing their school improvement plans. 
School Change Research 

Every culture builds on certain concepts, principles, or values that are 
fundamental to the individuals as a group within it, be it a school, a native tribe, or a 
large nation. These values are the sacrosanct entities that underlie all actions by the 
individuals and consideration of actions; they hold the whole "fabric" of the school or 
institution together. The concept of change, inextricably linked with the idea of 
progress, is seen as one of these values in education: as an almost automatic good, 
equating continual change with a seemingly inenviable progression toward perfection. 
These assumptions have been operant in the policy area of education, if not so 
frequent at the level of classroom practice. 

How, then, to transplant valuing and implementation of change to the 
classroom, so that improvement ensues? Research provides some illumination, thus, 
selections from the knowledge base on change follow. 

Hersey and Blanchard (1982) identified four kinds of educational change: 
change in 1) knowledge, 2) attitude, 3) behavior, and 4) group performance. Change 
becomes more time-consuming and energy-demanding, moving from 1 ) to 4). It is 
clear that knowledge, while it is necessary, is not sufficient to bring about behavior or 
performance change, although there continues to be an assumption that if information 
is just provided, then people (teachers) will understand the benefits and change their 



behavior. History has proven this untrue. 

Change is more likely to be successful when input is accessed from those who 
will be involved (Guskey, 1986; Ueberman & Miller, 1986). Further, when there is a 
need to which people can relate the change, change is more likely to occur (Jacullo- 
Noto, 1986). This axiom was proven at the demonstration schools as teachers learned 
to study and interpret data, and identify and prioritze needs. 

Too much change tends to be disruptive, therefore, there should be an 
adequate degree of stability to counter balance the impact of change, with a provision 
of stability and emotional reassurance (Ortiz, 1986). The leadership supporting 
change needs to be nurturing and supportive of those implementing change, thus 
leadership must contribute to and maintain a level of stability at the same time they are 
exercising intitiative for change. As Huberman and Miles (1986) reveal, 
"administrative decisiveness bordering on coercion, but intelligently and supportively 
exercised, may be the surest path to significant school improvement," (p. 70). The 
balance of pressure (initiating change, decisiveness) and support at the administrative 
leadership level is a delicate but necessary condition for effective change. 

As an organizational dynamic, change is a process and is highly individualized 
and based on personal experience (Hall & Hord, 1987). The process is one of 
developmental growth, and highly in need of facilitator focus on individual needs. Hall 
and Hord's research-based model provides for this perspective on the individual and 
the model's strategies address the inherent stresses in the change process. 
Consideration is given to the individual's personal reactions, the programmatic 
adjustments made by the individuals, their actual in-practice behaviors, and 
procedures employed by change facilitiators to support the process. The model's 
strengths are its client centeredness, addressment of individual concerns, stress on 
effective intervention by facilitators, maximization of effect, and minimization of 
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frustrations. 

Fuilan (1985) also stresses the importance of a facilitator to assist the school in 
the implementation process. In addition, Fuilan emphasizes the need to allocate 
sufficient resources (money and time), develop the principal's leadership role, provide 
ongoing staff development and assistance, and summarily, review the school's 
developing capacity to change - as the latter is an over-riding and essential goal of the 
efforts. 

Obviously these modest selections from the results of studies of change 
represent only a small percentage of that knowledge base. To varying degrees, the 
notions provided by these researchers and others were incorporated into the planning 
implementation of change at the demonstration sites. 

Educational Indicator Systems 

Educational indicators are defined as a single or composite statistics reflecting 
the health of the educational system, that can be readily, reliably, and repeatedly 
obtained, thus permitting comparisons over time and among states or districts 
(, Johnstone, 1981 ; Kaagan & Smith, 1985; National Center for Educational Statistics, 
1985). An indicator system is a framework into which any number of indicators are 
placed for review and analysis, leading to modification of policy and practice (Kaagan 
& Coley, 1988). The value of an indicator is in its ability to demonstrate change over 
time, the relationship of performance of one school district to another, and how the 
condition it measures compares with societal needs or expectations (Kaagan & Coley, 
1988). Oakes (1989) has suggested that the very existence of indicators influence 
how schools perform. Indicators have been suggested as guides for educational 
imorovement and as a measure of accountability. This paper focuses on the use of 
indicators as guides for school improvement at the district level. 

Interestingly, indicators have been justified throughout our nation's history, not 



only as a mechanism for reflecting the nature of educational problems, but as an 
instrument of educational reform (Shavelson. McDonnell & Oakes, 1989 ). The 
Common School Movement is an example of an early indicator system (Travers, 
1983) in which indicators served as justification for the establishment of the U.S. 
Department of Education in 1867 (Warren, 1974). The indicator movement in 1867 
was in response to problems in our nation's schools at that time much like the indicator 
movement of today. 

The current interest in educational indicators has developed rapidly over the 
past five years as local, state, and national agencies have been under pressure from 
the public to improve the quality of education. The national level of concern has been 
characterized by the nation's growing concern about literacy and the future ability of 
U.S. students to compete in the world arena. Indicators are also seen as one way to 
meet the needs of state and federal policymakers for comparative data on the quality 
of education in the U.S. (David, 1987). 

Indicators gained national recognition in November, 1984, when the Council of 
Chief State School Officers initiated a policy to establish a national system of 
standardized indicators; the council has since created a center to implement their 
policy (Kaagan & Smith, 1985; Smith, 1988). The U.S. Department of Education, The 
National Science Foundation, and the Council of Chief Stale School Officers gather 
and report cross-sectional or longitudinal information about student achievement and 
educational conditions in a representative sample of schools. This information can be 
used to draw conclusions about schools on a national basis. Federal agencies believe 
that indicators are essential for monitoring the status of the nation's educational 
system and for tracking how it changes over time. Unfortunately this information has 
not been designed to provide information to an individual state or district indicator 
system. 
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Many individual states are using statewide indicators to assess school 
improvement and the impact of educational reforms (Seldon, 1987, 1988). State 
level agencies believe that indicators provide information that can be used to hold 
local districts and schools accountable for their performance and provide data that can 
inform new improvement efforts ( Oakes, 1989). Some states have developed 
indicator systems for evaluating and comparing local districts within the state; 
individual districts too are developing their own sets of indicators to measure their 
effectiveness and efficiency with school and district specific indicators (David, 1988). 
The state and local level agencies have concerned themselves with such issues as 
monitoring policies, teacher certification, dropout rates, student achievement and 
increased academic requirements for graduation. These concerns will require a much 
more sophisticated measure of the processes and outcomes of schooling. 

It has become evident that to date a comprehensive indicator system is not in 
place to measure the outcomes of our educational systems. The difficulties in creating 
such a system are very complex including the need for common definitions, measures 
that match and adequately reflect education goafs, and methods for insuring fairness 
of comparisons across different populations (National Center for Educational 
Statistics, 1985; Oakes, 1988). What we do have are instead many attempts across 
the nation to develop general indicator systems along with state and local indicator 
systems. 

Why the sudden attention to indicator systems? Why the interest in developing 
a comprehensive set of national, state and local indicators? One explanation might 
be that if a comprehensive indicator system were available, policymakers could 
determine the nature of current and emerging problems, evaluate the factors 
influencing educational trends, monitor the effects of policy, and identify steps that 
might be taken to improve student performance (Shavelson, McDonnel & Oakes, 
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1989). As educational reforms are implemented, the indicator system could assess 
their contribution to better schooling (Oakes, 1986). Changes within the system for the 
betterment of the school could then be based on indicators that are tied to specific 
schooling processes along with the analysis of data relevant to future policy issues. 

Project and Regional Overview 

The goal of the Theme D project entitled, "Improving School and Classroom 
Productivity," was to develop strategies for school improvement based on an R a D 
school improvement process in five low income, rural, resource bound, demonstration 
sites. These demonstration schools, one in each of the five states served by 
Southwest Educational Development Laboratory, were selected on the factors 
mentioned above. 
Project Description 

In these five multi-year projects, the activities and events related to site 
selection, negotiation for work at the site, planning for the school improvment project 
and implementation of campus imrovement plans have been documented by a 
specified SEDUIiaison/extemal assister/change facilitator responsible for 
spearheading each school's improvement process. The participant observer made 
regular monthly on-site visits, conducted and attended school planning meetings, 
solicited feedback from teachers and administrators and made regular weekly 
telephone contacts with a wide variety of school and district personnel. A 
representative from each schools' state department of education, who served as a 
partner in the schools' improvement effort, provided additional information and 
assistance in each project's efforts. 

The effective schools research, school improvement process and school 
change research mentioned previously served as a knowledge base for all involved in 
the Theme D effort. Extensive training was provided to each of the sites in these areas 

10 



12 



along with extensive on site leadership development. This leadership development 
was in the form of: training; modeling and coaching; developing short and long range 
plans; networking; and other professional development acitivites. The leadership 
training was targeted not only at the superintendent, principal and key central office 
staff but at master teachers and teacher leadership teams. 

Overall project indicators were developed by SEDL at the onset cf the project. 
These indicators were to serve only as a guide for the project with the expectation that 
each site would develop its own site spec s fic indicators from these general guideline 
indicators. These general project indicators are as follows: 

Svstam Indicators; 

1 . Evidence that tests are being used for diagnosis as wail as measurement. 

2. Opportunity for teachers to collaborate exists within the system. 

3. Partners and other agencies and individuals are being regularly accessed as 
resources. 

4. Schedules reflect knowledge of research on time on task. 
Client Group Indicators: 

1 . Evidence that student achievement is rising. 

2. Evidence that teachers understand and use research-based information on 
effective schools, effective teaching and change processes. 

3. Evidence that the staff are assuming leadership roles both inside and outside 
the school. 

4. Evidence that the principal is serving as a,i instructional leader. 
Decisionmaking Indicators: 

1. Decisions specific to each sites' distinctive needs are bding made by people 
who will need to implement the decisions. 

2. Decisionmaking is data-based. 
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3. Decisionmaking is grounded in instructional/learner needs. 

4. Decisionmaking group meets regularly to analyze data and make 
instructional decisions based on this data. 

Since comparison data are not available, the above project indicators served as one 
way, not the only way to access educational reform typified by the entry conditions 
described previously. 
Demographics of the Region 

It is important to understand the vastness and the extremes in the region served 
by SEDL to fully understand the context in which this study took place. Each site 
brings to the study its own uniqueness which is directly influenced by its state 
demographics and state educational context. 

The five states of the Southwest region comprise an area of great diversity. In 
landscape and climate, in people and cultures, In population density, and in natural 
resources and economies. The region includes: topography ranging from deserts to 
semitropical coastlines; some of the nation's most densely populated and most rural 
areas; some of the nation's richest and poorest counties and schools; some of the 
nation's highest and lowest percentages of Black, Hispanic and American Indian 
students in public schools; and some of the highest percentages of public school 
students classified as limited-English proficient (Vaughan, et al., 1989). 

The region's schools enroll more than five million students from kindergarten 
through the twelfth grade. Texas has approximately 65% of the total population and 
public school enrollment in the region. New Mexico, with more than a fifth of the 
region's land area accounts for only five percent of the region's population and school 
enrollment. Louisiana is the most densely populated state in the region and has nine 
times as many people per square mile as New Mexico. Arkansas is the most rural 
state in the region with 48% of its residents living in rural areas; Oklahoma has 33% of 
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it's population in rural a**as (Vaughan, 1989). 

The five states have significant minority populations. New Mexico and Texas 
have high proportions of Hispanic residents. Oklahoma and New Mexico have high 
proportions of American Indian residents including Apache, Cherokee, Comanche, 
Pawnee, Navajo, Pueblo, and Zuni. Arkansas, Louisiana and Texas have relatively 
high percentages of black populations. Texas also has a growing Asian-American 
population (Vaughan, 1989). 

Public Education 

Beginning in the early 1980's the states in the region initiated a wide range of 
educational reforms and school improvement efforts and incentives. These included: 
mandated teacher and administrator appraisal processes; career ladder programs; no- 
pass no-play requirements; curricular mandates for basic skill instruction; remediation 
programs for districts whose students perform poorly on standardized tests; district 
accountability measures and progress reports; consolidation regulations and 
statewide effective school projects. 

Arkansas. Just over 40% of the population in the state of Arkansas live in rural places 
with fewer than 2500 inhabitants. There are centers of rapid growth near Little Rock 
and in the retirement communities close to the Ozarks and Hot Springs National Park. 
Of the 329 independent school districts in Arkansas, at least three-fifths are designated 
as rural or mostly rural. About two-thirds of the districts enroll less than 1000 students 
(Vaughan, 1989). 

An ambitious education reform agenda has focused most strongly on: increased 
standards for curriculum and staff certification; formal accountability through annual 
accreditation; ;aacher and student competency testing; and increased school 
attendance and parent involvement. There are also mandates for consolidation of 
school districts that fail to meet or maintain accreditation and certification standards. 
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Early studies indicate improvement in education and observers are positive about the 
overall resu!ts (Center for Research and Public Policy, 1988; Root, 1989). 
Louisiana. Louisiana has approximately 4.4 million people, with 800,000 students in 
the public schools. Laws are derived from the Napoleonic Code and the counties are 
called parishes. Louisiana schools are orgaized into 64 parishes, the smallest having 
1500 students. Three urban parishes (Orleans, East Baton Rouge, and Caddo) plus 
suburban Jefferson Parish enroll more than 20% of the public school students 
(Vaughan, 1989). 

During the 1988 Legislative session, the Children First Reform Legislation was 
signed into law representing a serious commitment to improving the quality of 
education in Louisiana. The act proscribes the development of a performance-based 
teacher appraisal system, a data collection system for educational indicators, and a 
reorganized system of intermediate regional service centers. The law requires 
extensive consultation with administrative and stakeholder groups in the design and 
implementation of the teacher appraisal and indicator systems (Market Research and 
Issues Management, Inc., 1989; Louisiana Department of Education, Office of Reform 
Coordination: Management and Oversight, 1989). 
New Mexico. When New Mexico was admitted to full statehood, its first state 
constitution made bilingualism official in the legislature and in the schools. People of 
Spanish origin are approximately 35% of the population and American Indian are 
approximately 8%. There are currently 88 school districts in the state varying from 60 
to 79,850 students. Of these, 32 districts enroll fewer than 500 students. 

New Mexico has adopted a number of reform initiatives but tho provisions have 
been phased in over a period of tme rather than in reform initiative. These reform 
elements include student performance standards, class size restrictions and new 
curriculum requirements. The state agency also encourages local schools in rural 
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areas to form voluntary networks for planning staff development and school 
improvement. The educational reform actions have increased expectations for 
students and districts. 

Oklahoma. Oklahoma is primarily a rural state, with only three metropolitan areas, 
Tulsa, Oklahoma City and Lawton. The rest of the state includes small towns and 
villages. There are 611 school districts. Approximately three-fourths have less than 
1 000 average daily attendance. Thirty-eight percent of the districts enroll less than 
250 students. 

The "Oklahoma 2000 Education Challenge Act" was passed by the 1989 
Legislature and included a variety of measures designed to improve the quality of 
education in the school system. The act requires the State Board of Education to 
establish an Oklahoma Educational Indicators Program to assess and report the 
performance of public schools and school districts. Indicators of progress will be 
developed for all grades and applied so that educators, students, parents and 
communities will receive information about school performance (Oklahoma State 
Senate, 1989 regular session; Oklahoma Educator, 1989). 

Texas. Texas' land mass, population, and number of school districts present a diverse 
school population. Texas is growing in size and heterogeneity and has a projected 
school population of approximately 3.5 million students and increasing proportions of 
ethnic and racial minorities. There are 1063 school districts in Texas, although the 
largest eight districts enroll 20% of the state's total enrollment. In 1988, the state 
government spent $12.85 billion on public schools - kindergarten through twelfth 
grade (Blackstone, 1989; Vaughan, 1989). 

The Long-Range Plan for Texas Public School Education was mandated by 
House Bill 72, the education reform legislation enacted by the 68th Texas Legislature. 
The goals are related to: student performance, curriculum, teachers and teaching, 
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organization and management, finance, parent and community involvement, 
innovation and communication. A variety of programs were designed to ensure the 
successful implementation of education goals (Texas Education Agency, 1987). 
Description of Demonstration School Silas 

School A is located in a small town of approximately 4200 people which 
exemplifies small town America. The main agricultural crops are rice, cotton, and 
soybeans. In recent years fish farming has become a major source of income with 
over 20,000 acres devoted to fish being raised locally each year. Local retailing 
business offers other employment opportunities while many residents commute daily 
to jobs in a major city 30 minutes away. 

Four schools make up School A's district (student population 1780); a high 
school (351); a junior high school, grades 7-9 (433); an elementary school, grades 4-6 
(426); and a primary school for kindergarten through third grade (570). The 
laboratory project was involved directly with only the elementary and primary schools. 
The student population is 75 percent white and 25 percent black. Approximately 53 
percent of the students are transported. About 40 percent of the students are eligiible 
for free or reduced lunches. The staff of 100 plus certified teachers is very stable with 
little turnover and a majority of the teachers living within the district and community. 

School B is located in a town of about 23,000 residents which borders an indian 
reservation. The major business is trading indian art. The town has an extension of 
the university, many restaurants, hotels, and shops catering to the tourism trade, and 
various federal service agencies for the Native Americans. 

The school district covers the largest of the five studied, covering a vast area 
spread over rough, sparsely populated terrain. The district serves 13,000 students in 
their 27 schools. There are 17 elementary schools (eight are in town while the 
remaining nine are out of town, some as far away as 56 miles) and four middle 
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schools, two in town and two out of town. Only one of the high schools is in town with 
five located out of the town. 

The district's student population is 68% Native American, 16% Hispanic, and 
16% white. School B's population is 58% Native American, 21% Hispanic , and 21% 
Anglo. It is one of the in-town schools, but does have students that are living outside 
the town boundaries. 

School C is located in a small village of 450 people. There is a limited 
economic base, with two handy stop grocery /gasoline pump businesses and a 
package store. Peanut farming has been the agricultural producer in the area of 
gently rolling hills. There are three churches in the town. 

There are two schools in the district: elementary, grades k-6 (120 students) and 
secondary, grades 7-12 (110) students. The student population includes Anglos, and 
Chickasaw and Choctaw Native Americans. Over half of the children come from single 
parent families. Many of the students participate in a breakfast program with 60% of 
the children also participating in the free or reduced lunch program. 

School D is located in a community of approximately 6,000 and is located in 
one of the poorest districts in the five state region. The economic base is limited with 
approximately a dozen businesses (restaurant, florist, hardware store, etc. ) 
comprising the small "downtown area." Dairy farming has been the leading 
agricultural producer in the area for many years along with the timber industry. 

At this time there are three schools in the district: elementary, grades PreK-4 
(800 students); middle school, grades 5-8 (630 students); and a high schoo? (440 
students). The student population is primarily black. 80% of the children are on free or 
reduced lunch programs with many students also participating in the breakfast 
program. A large majority of the students ride buses to isolated areas of the district. 
The school district employs 275 people. 
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SfibflOLE is located in a very small town of about 1 ,000 residents of which the 
majority are Hispanic (95%) located very close to the U.SVMexican border. 
Businesses include a video tape store, a trading post, a quick snack shop that has 
recently expanded to include a cafe, and a modern cotton gin. 

The school district has approximately 400 students. There are two schools: the 
elementary school (grades K-6) housed in an old building and the high school 
(grades 7-12) which is housed in a modern twenty year old building. The student 
population reflects that of the community (95% Hispanic). Most students (70%) are 
Spanish speaking with limited English proficiency. 

Results of the Study 

Data Analysis 

Data analysis in this paper includes quantitative information which includes test 
score information in reading, mathematics and language arts, and qualitative 
contextual indicator information. For some schools, achievement was analyzed on 
both norm referenced and criterion referenced tests when both types of test 
information were available. 

A series of tables report the data. Not all schools (D & E) reported test data due 
to particular circumstances at these sites (this information is expected late spring). One 
school (C) reported only aggregate school data on a norm referenced test. Each table 
varies depending on which nomn referenced test was reported and at which grade 
levels the test was given. Each table presents the grade level for which the data are 
reported, the baseline data (86-E7), the current results data (88-89) and the gain or 
loss between baseline data and current results data. 
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School A 





Norm Referenced Test 




Math* 


86-87 


88-89 




.Grade Level 


Baseline Data 


Results Data 


Gain 


1 


70 


71 


.i 
♦ i 


2 


67 


82 


+15 


4 


67 


72 


+5 


5 


63 


74 


+11 


* Reported in percentile scores 














Norm Referenced Test 




Language* 


86-87 


88-89 




Grade Level 


Baseline Data 


Results Data 


Gain 


1 


61 


74 


+13 


2 


63 


72 


+9 


4 


56 


61 


+5 


5 


57 


58 


+1 


•Reported in percentile scores 








Norm Referenced Test 




Reading* 


86-87 


88-89 




Grade Level 


Baseline Data 


Results Data 


Gain 


1 


66 


74 


+8 


2 


54 


58 


+4 


4 


56 


54 




5 


57 


56 
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Loss 



Loss 



Loss 



■2 
-1 



0 

ERIC 



It is clear that in School A significant gains were found in math and language in grades 
one, two, four and five on the norm referenced test. Impressive gains in first and 
second grade language scores are of importance. Significant gains were found in first 
and second grade reading scores. A loss of up to two percentile points was found in 
fourth and fifth grade reading scores. In most instances, the improved performance 
increase was signficant- as high as fifteen percentile points. 



ScbfiflLB 

Norm Referenced Test 

Math* 86 - 87 88 - 89 

Grade Level Basalina Data Results Data Gain JLQSS 

K 47 53 +6 (13%) 

1 48 66 +18(38%) 

2 48 59 +11(23%) 

3 46 49 +3(7%) 

4 59 65 +6 (10%) 

5 45 53 +8 (18%) 



* Reported in NCE Scores 

(continued) 
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Norm Referenced Test 



Language* 86 - 87 88 - 89 

Grade Level Baaalina Data Bflsutta Data Gain Loss 

K 

1 51 61 +10(20%) 

2 46 57 +11(24%) 

3 46 51 +5(11%) 

4 57 55 -2(%4) 

5 45 49 +4(8%) 
•Reported in NCE Scores 

Norm Referenced Test 

Reading* 86 - 87 88 - 89 

Grade Level Baseline Data Results Data Sain LOSS 

K 45 52 +7(15%) 

1 45 59 +14(31%) 

2 45 57 +12(21%) 

3 44 50 +6(14%) 

4 50 53 +3(6%) 

5 50 53 +5(11%) 



* Reported in NCE Scores 
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It is clear that student achievement increased in reading, language arts and math at all 
grade levels except in the fourth grade language scores (-2 NCE). The most dramatic 
gains were seen in the first and second grade scores. Here total aggregate gains in 
reading, language and math averaged a 26% increase across the three year period of 
time. The improved performance increase was significant in most cases - as high as 
18 NCE points. Criterion referenced test information was not available for School B. 
ScbflQLC 

Norm Referenced Test 
Total School* 86-87 88-89 

Subject Baseline Data Results Data Gain l£S& 

Reading 44 57 +57 

Language 41 43 +2 

Math 45 85 +40 

* Reported in percentile scores 

School C reported aggregate test results for the total school on a norm referenced test. 
Total school student achievement increased significantly in reading and language and 
most dramatically in math (+40 percentile points). 

In conclusion, where quantitative data (test results) are available in schools A, B 
and C, significant increases in reading, math and language arts were found. The 
reader should consider that two of the schools (A & C) used the same norm referenced 
test while School B used a different norm referenced test. Comparisons between sites 
is difficult because of this difference, coupled with many other site specific differences 
(school size, input resource variables). 

22 



24 



Contextual Indicators 

One goal of a comprehensive indicator system is to develop and provide 
different types of measures of student achievement and school success. The state of 
the art in measurement and evaluation has grown extensively in recent years. 
However, measurement technology and the art of assessment are still in their infancy. 
Sometimes what needs to be measured cannot be done due to the lack of a proper 
traditional measurement tool. This has many implications for measuring school 
improvement efforts. 

To compound the problem, schools develop multiple goals and objectives and 
then measure the multiple goals with a single set of outcome measures (Oakes, 1989). 
The goal of a comprehensive indicator system should be that of developing different 
measures of school success to then drive the school improvement efforts. 

Oakes (1989) has suggested context indicators as one way of capturing the 
total picture of school performance. Context indicators provide information abort the 
central features of the educational system (Oakes, 1989) such as resource levels, 
policies, administrative suppport and focus on academics. Policy makers and 
educators are concerned with issues in the school such as school climate, parental 
involvement, time on task, quality of resources, instructional focus and school 
leadership along with the common outcome indicator of student achievement. The 
former is more difficult to measure and describe but no less important. The above 
mentioned contextual indicators are referred to by Oakes (1989) as enabling . 
indicators: they facilitate or enable learning to take place in the school. 

Context indicators also enable the dissaggregation data by subgroups that 
have had like experiences in school (Oakes, 1989). A complete view of must be in 
place to fully understand the outcome indicators and to then decide on areas needed 
for improvement. That is, one must consider input indicators, process indicators and 
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output indicators within the realm of contextual Indicators to completely grasp the total 
picture of a particular educational setting. By using test-score indicators with equally 
balanced and respected context indicators, a complete picture of school effectiveness 
can be achieved. 

Demonstration School Development of Contextual Indicators 

The development of contextual indicators poses a dilemma. For schools to 
decide on what to include in their indicator system, they must first understand what 
qualities of teaching are most needed for successful learning and what characteristics 
make the school "effective." SEDL provided to each of the five schools training and 
on-going staff development in the following related areas: effective schools research; 
school improvement process; school effect/effective teaching practices; team building; 
parental involvement; indicators; and other related topics such as communication, 
behavior management, test taking strategies, at-risk issues, time on task, expectation; 
and homework. Extensive training in leadership both on and off site was provided to 
administrators and teacher leaders. This information and training provided a common 
knowledge base for the five schools. 

In addition to the training and staff development, SEDL provided continual 
assistance to the districts in the form of an external change facilitator/site liaison. This 
person provided continual guidance and support to the district in developing: short 
and long term goals; action plans; site specific strategies for school improvement; and 
site based indicators of success. More importantly, this change facilitator developed, 
over time, a vision with the school of what the Ideal" school situation would be for their 
particular site. This was the initial step, and a most important step, in the development 
of site indicators • the clear vision. The schools then outlined their vision in 
relationship to their school area of focus for improvement. Given the diversity of the 
schools, the focus areas were just as diverse ranging from writing/technology/critical 
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thinking skills to school climate to reading instruction to at-risk and slow learner 
strategies. Indicators were developed individually with each site with the site change 
facilitator. These site indicators were then categorized by the change facilitator and a 
representative from the particular school site under the Oakes (1989) construct of the 
following enabling contextual indicators: access to knowledge; press for achievement; 
and school professional conditions. The following is a brief review of contextual 
indicators. These indicators represent changes in each site that have occured over the 
last three years as a result of the improvement effort 

School A: Focus Area - At-Risk. Special Needs Populations 

Access to Knowledge 

1 . Resource teachers are used as specialists to provide information to regular 
teachers concerning mainstreaming special needs children. 

2. Teachers have received training in coooperative learning end have utilized 
heterogeneous groupings throughout most of the school for all students. 

3. Academic support system for all students is available. Homework Centers 
and peer tutoring are available for all students. 

4. Parents work as instructional aides in the classroom. 

5. A 30 minute planning period each morning is available to all teachers to meet 
and plan together. 

6. Faculty is focused on the effective use of instructional time. 

7. Availability of instructional materials along with computer laboratories 
increase. 

8. Teachers receive extra monies for instructional use due to their own creative 
grant writing. 

9. Adoption and institution of a net? reading program K-2. 
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Press for Achievement 

1 . Entire staff emphasis is on student achievement and this is communicated 
throughout the school and community. 

2. School recognues academic accomplishments on a regular basis. 

3. All students are expected to pass the state basic skills tests. 

4. New ideas are encouraged by the administrative staff. 

5. Implementation of new instructional strategies is supported by the 
administrative staff. 

Professional Conditions 

1 . Release time is provided for professional interactions on a regular basis. 

2. Leadership teams meet on a regular basis. Effective School correlate 
committees meet regularly and involve all members of the faculty in the 
planning process. 

3. School-wide decisionmaking concerning specific at-risk students is evident 
in the Teacher Effectiveness Teams, organized to address individual at-risk 
student problems. 

4. Data is reviewed annually. Planning for the next school year is based on 
data driven information. 

School B: Focus Area: Writinafrechnaloav/Critical Thinking 
Access to Knowledge 

1. Teachers within the school have received training in the focus areas and now 
serve as "experts" to other faculty members both within the school and within 
the district. 

2. Innovative programs have expanded to encompass the focus areas: piocess 
writing; peer editing; Writing to Read; critical thinking programs. 

3. There is mobility from one ability group to another. School is moving from 
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ability grouping to mixed grouping. 

4. The computer lab is available to all students on a regular basis. The 
computer committee is forming a computer curriculum. 
Press for Achievement 

1 . Academics are emphasized which is evidenced by the staffs increased 
knowledge in specific areas (writing, technology, critical thinking). 

2. A majority of the students participate in classes stressing critical thinking 
skills. 

3. Academic excellence is recognized regularly by specific awards for 
excellence and improvement. 

4. Praise is given from the "front-line administrator" to the teachers for 
implementing challenging curriculum into their classes. 

5. People are brought into the district from neighboring districts to visit the 
school. The school is used as a model of instruction for others to follow. 
Professional Conditions 

1 . Class size has been reduced. 

2. Committees meet on a regular basis. 

3. Internal Affairs CommHee has evolved to become a school decision making 
entity. 

4. Teachers have moved from approximately 10% to now 50% believing that 
they can make a difference with each student. 

5. Teachers who are taking risks (trying new things) are receiving school 
recognition. 

6. A significant number of parents have communicated to the school that their 
students look forward to each school day. 
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Schnnlfr Focus Area: Reading 
Access to Knowledge 

1 . Chapter I teachers moved from a reading focus to a language arts focus. 

2. Institution of a new reading program. 

3. Development of a student retention policy. 

4. Development of an elementary library. 

5. Institution of a remedial summer school program. 

6. Parental involvement evolved from virtually nonexistence to being a 
major component in the school. 

Press for Achievement 

1 . The entire staff is focused on academic achievement from the superintendent, 
who provides rewards and incentives for excellence, to the teaching staff. 

2. School-wide incentive programs for reading accomplishments are provided. 

3. The superintendent is involved directly with teachers in exploring new 
instructional programs. 

4. Teachers developed a homework policy, submitted it to the administration 
and the School Board for ratification and adoption as policy. 

Professional Conditions 

1 . Departmentalization is replaced by self contained classes as the 
organizational structure. 

2. Administrator teaches a class so planning period remains intact for all 
teachers. 

3. Significant amount of time is devoted to planning, collaboration and 
collegial planning. 
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4. Teachers acquired a new view of themselves as a "can do" professional. 

5. Administrative support for teachers' noninstructional tasks has increased 
significantly. 

School D: Focus Ama: School Climate 

Access to Knowledge 

1 . Coordinating teachers assist other teachers with instructional issues. 

2. Course offerings increase at the secondary level. 

3. A computer lab is made available through a special grant. 

4. Parents become involved in the school by: involvement in the 
school wide testing program; fund raising monies for specific school 
projects; and aiding instruction in the classroom. 

5. Staff believes that all children can leam and that they will do the best they 
can with what they have where they are. 

Press for Achievement 

1 . Administrative support by the superintendent and certain administrators 
provides the impetus for teachers to examine new programs. 

2. Homework policy instituted district-wide. 

3. Honor students are recognized district-wide. 
Professional Conditions 

1 . Teachers at all schools meet monthly in leadership teams to plan for school 
improvement. 

2. All schools have a new support - a full time school secretary. 
School E: Focus Area: Climate 

Access to Knowledge 

1. Two technical teachers to address language deficiencies added to the 
faculty. 
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2. Instructional time increased proportionally to the day. 

3. A language enrichment program added for ESL students. 

4. Primary limited English proficiency students placed with a bilingual teacher. 

5. Computers purchased and made available to supplement instruction. 

6. A local retailer with a reading incentive program provides an incentive for 
qualifying students. 

7. Parent volunteers recruited, trained and used throughout the school. 

8. School communicates with the parents through a monthly newsletter. 

9. District band teacher and band program added, with uniforms provided. 
Press for Achievement 

1 . School-wide intensive preparation for state mandated test. 

2. Satellite classes in Trigonometry and French along with language 
enrichment classes are a part of the course offerings. 

3. Honor students are recognized. 

4. Administration orders materials for programs in a timely manner. 
Professional Conditions 

1 . Teacher input provided concerning a reward change caused by budget 
adjustment. 

2. Teacher release time provided for inservice activities. 

3. A significant amount of time is spent in planning for school improvement. 

4. Teacher input provided concerning utilization of computers. 
Discussion of Contextual Indicators 

School A 

School A developed an elaborate instructional system to address the needs of 
students who were not experiencing success in school. These included 
mainstreamed special education, slow learner and at-risk populations. Of special 
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concern to School A was the reading level and language development of these 
populations and the entire K-2 student population. The initiation of the new reading 
program (School A/Access Indicator #9) has been suggested as one contributing 
factor in improving reading and language scores grades K-2. Special assistance 
given to regular educator* from special ecucators, cooperative learning, inservice 
traiing and academic support systems have been identified by the school faculty as an 
enabling factor in the general overall rise in test scores demonstrated on the norm 
referenced test. The improved school climate has been attributed to Press for 
Achievement indicators one through five. Teachers feel that all students will learn in 
their school because the necessary supports are now instituted into the school 
(examples: Teacher Effectiveness Team, Leadership Teams, annual data review 
process). 
School B 

School B focused its attention on the areas of writing, critical thinking and 
technology. Impressive gains on the norm referenced test were demonstrated in K-2 
scores in readir-, language and math. The school attributes these gains in part to the 
extensive use of the computer lab in teaching the fundamentals of reading, writing and 
language to the lower elementary aged students. Gains are less impressive in math, 
reading and language at the upper elementary level (grades 4-5). This has occured, 
they believe, because these students have not had the full benefit of the computer lab 
which focuses on reading, writing and language skills. The math gains demonstrated 
by the fourth and fifth grade students are in fact slightly higher than their language and 
reading gains. School B reports that a majority of teachers believe that all students 
are capable of learning higher level thinking skills and have incorporated this 
discipline into the whole curriculum. Of significance is the fact that teachers are now 
"taking risks" and "trying new things" and are being rewarded for their risk taking. The 
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School B adminsitrator feels that this risk taking and experimentation has changed the 
climate of the school significantly. 
School C 

Although School C reported only school-wide aggregate test scores on the 
norm referenced tests, a number of hypotheses can be made between the test scores 
and the contextual indicators. 

Significant gains were demonstrated in reading and math scores across the 
school. School C attributes this dramatic rise in scores to adoption of a new reading 
program K-2; instructional focus of staff and adminstration; staff change in the math 
department; and the institutionalization of a homework policy. 
School D 

School D has undergone an extensive district-wide reorganization which has 
had both negative and positive results on school improvement. A concern during the 
reorganization was that of school climate, teacher morale and school resources. Test 
scores from the reorganized district are not available until late spring. More 
importantly, although this district has experienced trauma, they have emerged with a 
new sense of focus and purpose. When examining School D's contextual indicators 
over the past three years, it is evident that significant changes have occured which are 
difficult to measure in the conventional manner. 

The lack of resources at School D has been a major concern throughout the 
project. School 0 has tried to remedy this problem by utilizing parents as a resource, 
applying for grants to supplement existing programs and by reorganizing the district. 
The reorganization of the district can be viewed as a general contextual indicator for 
School D since it impacted policymaking and decisionmaking in the district to a large 
degree. 
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School E 

School E also has undergone trauma related to a different type of school 
reorganization and has been impacted by this experience. As stated before, School 
E's test data will be reported in late spring and will be available in the final report in 
July 1 990. 

School E also chose school climate as a focus area due again in part to site 
specific situations and went about building school climate in many unique ways. The 
current superintendent played a major role in addressing the climate issues by: 
opening the lines of communication between the school and community; involving 
local businesres and parents as school partners; expanding the curriculum offerings; 
and rewarding faculty and students for excellence. Again, test information, now 
missing, may not reflect the dramatic changes in School E. 
Implications for Future Study 

Schools striving for increased effectiveness have needs for improvement that 
are simultaneously universal and unique. The school effects research has identified a 
set of variables that appear to be typically present in schools that are exceeding 
expectations in student success. At the same time, the school culture, context, and 
other factors demand attention to each school's particularistic needs - thus, "unique" 
needs. While student achievement outcomes are a universal standard against which 
to judge success or effectiveness, additional standards derive from each school's 
unique goals and objectives. How to accommodate these two sets of standards or 
indicators of educational success has been the focus of this paper. 

The results of this study, still in progress, are very encouraging. Data gathered 
on indicators of educational success from this project demonstrate that school 
improvement can be measured in a variety of ways that address individual school 
needs and reflect appropriately their specific gains, even under less than ideal 
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schooling conditions, with different levels of success. Some of these indicators 
include: the frequency of teachers' interactions about instruction; parental involvement 
activities in schools; increase in communication of teacher/teacher, 
administrator/teacher, teacher/parent, etc.; teacher-initiated policy development and 
implementation. 

Schools are complex social institutions that require multiple assessment 
criteria. Unfortunately, the most easily measured and categorized school information 
are most likely to be the least useful. The most important attributes of an effective 
school environment is in most cases the most difficult to measure in isolation. School 
improvement adds up to more than increased student achievement on standardized 
tests, though that is an undeniable primary goal; improving the school means many 
more things to each constituency involved. The whole school context must be 
continually be considered when assessing school achievement or school 
effectiveness. 

The question of what school characteristics are most important to assess as 
indicators is of the utmost importance. There is limited evidence concerning certain 
school characteristics and preferred outcome behaviors. Research is needed in the 
area of what to measure as school characteristics for indicators. Different 
characteristics within different schools produce different effects. More study is needed 
in this area to fully understand the complexities of the "whole school culture" and its 
interplay with achievement. 
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